import { Callout } from "nextra-theme-docs";

# Separated Deployment with Docker and K8s

Here we use the lightweight K3s as an example. If you have already deployed a K8s cluster, you can refer to the subsequent configuration for similar settings.

<Callout type="warning">

Please read the entire tutorial before proceeding, otherwise the deployment result may not meet expectations.

</Callout>

## Install Docker

You can find the installation instructions on the [Docker official website](https://docs.docker.com/engine/install/), and use the basic configuration in [Quick Start](/quick-start) to run GZCTF.

## Install K3s

K3s is a lightweight k8s distribution that can be quickly deployed on single and multiple machines. Official documentation: [https://docs.k3s.io/](https://docs.k3s.io/)

<Callout type="info">

If you only have one machine and want to run GZCTF through Docker, you can specify the Docker backend by adding the following parameters during installation:

```bash
INSTALL_K3S_EXEC="--docker"
```

<Callout to="info">

If you need to run more than 255 challenge containers on a single k3s instance, you **need to specify `INSTALL_K3S_EXEC` during k3s installation** and change `node-cidr-mask-size` to the desired subnet size.

```bash
INSTALL_K3S_EXEC="--kube-controller-manager-arg=node-cidr-mask-size=16"
```

**The above configuration CANNOT BE CHANGED after installation.** Changing the CIDR to `/16` can support 65535 Pods.

The above configuration only changes the IP address range used by the node Pods. If you need to change the node's Pod limit, please refer to the following.

</Callout>

And install k3s in the following way. For more information, please refer to [k3s installation configuration](https://docs.k3s.io/installation/configuration):

```bash
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="..." sh -
```

But this is not recommended, using Docker as a container backend may cause some functions to be unavailable.

If you don't want to use Docker as a container backend, but need to deploy on a single machine, you can refer to the method in [K8s Cluster Deployment](/deployment/k8s-only).

</Callout>

```bash
curl -sfL https://get.k3s.io | sh -
# Check for Ready node, takes ~30 seconds
sudo k3s kubectl get node
```

<Callout type="info">

Chinese users can use a mirror site to speed up installation.

```bash
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -
```

</Callout>

For multi-node installation and cluster setup, please refer to the [official documentation](https://docs.k3s.io/en/quick-start).

## Configure GZCTF

The connection configuration file for k3s is located at `/etc/rancher/k3s/k3s.yaml`, and it can be exported using the following command:

```bash
sudo cat /etc/rancher/k3s/k3s.yaml
```

Use the following command to obtain the IP address of the k3s control-panel machine:

```bash
sudo k3s kubectl cluster-info
```

<Callout type="info">

If it shows `127.0.0.1`, it means that the k3s control-panel is on the current machine. Please use `ip a` to check the IP address of the current machine.

You can directly use the IP address or use a domain name, but make sure the domain name resolves to the machine where the k3s control-panel is located, and ensure that the machine running GZCTF can access port 6443 on it.

</Callout>

Save the above output as `kube-config.yaml` and change the `server` field to the IP address of the machine where the k3s control-panel is located, for example:

```yaml
apiVersion: v1
clusters:
  - cluster:
      certificate-authority-data: # ...
      server: https://127.0.0.1:6443 # change this to your k3s control-panel's IP or domain
    name: default
# ...
```

Save it to the machine where GZCTF is deployed, in the same folder as `compose.yml`, for example, `kube-config.yaml`.
Then modify the mount configuration in `compose.yml`:

```yaml
gzctf:
  image: gztime/gzctf:latest
  restart: always
  ports:
    - "80:8080"
  networks:
    default:
  volumes:
    - "./data/files:/app/files"
    - "./appsettings.json:/app/appsettings.json:ro"
    - "./kube-config.yaml:/app/kube-config.yaml:ro" # this is required for k8s deployment
    # - "/var/run/docker.sock:/var/run/docker.sock" # this is required for docker deployment
  depends_on:
    - db
```

Also, change the `appsettings.json` file and set the `ContainerProvider` field:

```json
{
  "Type": "Kubernetes",
  "PublicEntry": "ctf.example.com" // change this to your k3s control-panel's IP or domain
}
```

Restart GZCTF, and then you can use k3s as the container backend. Users who have already used k8s can also refer to the above configuration process to integrate GZCTF into an existing k8s cluster.

## Change NodePort Port Range

The default NodePort port range for k3s is 30000-32767, which may not meet your requirements. Therefore, you can modify the NodePort port range of k3s according to your needs.

Run the following commands on the machine where the k3s control-panel is located:

- `sudo nano /etc/systemd/system/k3s.service`
- Edit the `ExecStart` setting below to specify `service-node-port-range`

  ```bash
  ExecStart=/usr/local/bin/k3s \
      server \
      --kube-apiserver-arg=service-node-port-range=20000-50000
  ```

- `sudo systemctl daemon-reload`
- `sudo systemctl restart k3s`

## Change the container limit of K3s

The default container limit of K3s is 110, which may not be suitable for a large number of small containers in a competition. Therefore, you can change the container limit of K3s according to your needs.

<Callout type="info">

Reference: [How do you increase maximum pods per node in K3S?](https://stackoverflow.com/questions/65894616/how-do-you-increase-maximum-pods-per-node-in-k3s)

</Callout>

Run the following commands on the machine where the k3s control-panel is located:

- `sudo nano /etc/rancher/k3s/kubelet.config`
- Edit the `maxPods` setting as follows

  ```yaml
  apiVersion: kubelet.config.k8s.io/v1beta1
  kind: KubeletConfiguration
  maxPods: 500
  ```

- `sudo nano /etc/systemd/system/k3s.service`

- Edit the `ExecStart` setting below to specify `kubelet-arg`

  ```bash
  ExecStart=/usr/local/bin/k3s \
      server \
      --kubelet-arg=config=/etc/rancher/k3s/kubelet.config
  ```

- `sudo systemctl daemon-reload`
- `sudo systemctl restart k3s`

## Add Container Image Registry

Using an external container image registry directly is not supported in k3s. You need to add the image registry to k3s.

Run the following commands on the machine where the k3s control-panel is located:

- `sudo nano /etc/rancher/k3s/registries.yaml`

- Edit the `mirrors` setting below to specify the address of the image registry you need

  ```yaml
  mirrors:
    "container.ctf.example.com": # change this to your registry's domain
      endpoint:
        - "https://container.ctf.example.com" # change this to your registry's domain
  ```

- `sudo systemctl restart k3s`
